Managing letterbox displays

ABSTRACT

A video display control system, comprises a video display having a first format display ratio. A picture height circuit determines an active video picture height from an input video signal having a second format display ratio. A detector circuit identifies letterbox formats responsive to the active video picture height in the video signal and determines a format display ratio of the letterbox picture. A zoom control circuit is operable in a first mode of operation for enlarging the picture in size to fill the display substantially entirely, notwithstanding consequent cropping of the picture, and operable in a second mode of operation for enlarging the picture in size to substantially fill the display vertically, notwithstanding consequent unused portions of the display. A vertical pan control circuit automatically centers the picture in both modes of operation. The detector can identify the format display ratio of the letterbox format picture a circuit responsive to the identified format display ratio controls image aspect ratio distortion of the enlarged picture. A deflection system is controllable in vertical size by a variable vertical scan rate, in horizontal size by variable horizontal video expansion and compression, and in pan position by varying the vertical reset in phase.

This application is a continuation-in-part of copending internationalapplication no. PCT/US91/03739, filed May 29, 1991, and designating theUnited States.

The invention relates to television receivers having a viewing area witha particular display format ratio or aspect ratio, including acontroller for determining the active video portion of a signal andadaptively displaying the active video portion so as to make full use ofthe vertical viewing area. The picture can be zoomed vertically andhorizontally. However, insofar as the zoom is such that the horizontaldimension does not correspond to the horizontal screen dimension, thepicture will be horizontally cropped or displayed with verticalsidebars. The invention responds adaptively to changes in the formatratio of the active video to selectively adjust the proportion ofcompression or expansion (zoom) and pan, to fill the entire viewingarea, or to vertically fill the viewing area, with the active videoportion of the received signal. The selection can depend upon viewerpreference or be programmed in advance. In one alternative, the viewerwatches a picture which fills the screen, at the cost of partiallycropped information content. In the other alternative, the viewerwatches a picture which has complete information content, at the cost ofwatching a smaller picture which does not utilize the entire screen.

The display area of a television receiver typically has either a formatdisplay ratio of 4 units of width to 3 units of height (referred to as"4×3" or "4:3"), or a format ratio of 16×9 (16:9), which isconventionally considered a "wide screen" or movie format ratio. Theseratios are relatively standard, although other ratios are known as well.Receivers having a particular format display ratio are limited in theways that signals provided at a different format ratio can be presented.Similarly, where multiple video signals are to be displayed, the formatratio available for a given one of the signals may present constraints.

Commercial broadcasts are most often in a 4×3 format display ratio(relatively taller and/or narrower), such that the full display area ofa conventional 4×3 receiver is occupied by the full picture. Whereasmost video products on the market have this 4×3 ratio, wide screendisplays are also known, intended for example for viewing wide screenmovies. Movie productions are available in widely varying aspect ratios,and the 16×9 ratio (relatively shorter and/or wider) is more or lessstandard for a wide screen format for television.

Many viewers find the 4×3 display format less pleasing than the widerformat display ratio associated with movies. Regardless of where thereceiver may be, the wider format is perceived to give the impression ofa movie theater while the narrower format has a look associated byviewers with home television viewing. A wide format display uses itsfull area to display the full movie ratio signal, without the need tocrop or distort the original movie picture, e.g., via a telecine device,processors in the television receiver, or the like.

To show a 4×3 signal on a 16×9 display unit, or to show a 16×9 signal ona 4×3 display unit, either less than all of the display unit area isused, or the video information is altered. The received picture can bezoomed to fill the screen in one dimension, with portions in the otherdimension removed from the signal. For example, top and bottom portionsof a 4×3 signal can be cropped, with the remainder filling a 16×9 formatarea, or side portions of a 16×9 signal can be cropped, with theremainder filling a 4×3 area. It is the ratio of width to height whichis of concern rather than any need to enlarge or contract the signal ingeneral.

Instead of simply cropping the signal, it is known to pan up and down orfrom side to side, either automatically or under control of a telecineoperator or the like, to avoid loss of important information in thescene. It is also known to distort the signal to be displayed, forexample compressing a 16×9 signal horizontally for display on a 4×3display unit, as is often seen when screen credits from a Cinemascope(16×9) movie are displayed in a commercial (4×3) broadcast. Cropping,panning and distortion all omit or adversely affect the quality and/orcontent of the picture.

When displaying one format ratio active video signal without cropping ordistortion on a different format ratio display unit, the displayedsignal occupies less than all the available area of the screen. Theunused area of the display unit is blanked or caused to display abackground color matte. However, this border area can be used, e.g., todisplay another picture simultaneously with the main picture, or todisplay text information.

In so-called "letterbox" format, a wide format ratio image is displayedacross the full width of a narrower format ratio display unit, and thetop and/or bottom areas are used if at all to display text such as stormwarnings, news alerts, etc. This function can be made selectable at theviewer's option. Movies in a wide format to be broadcast commerciallywithout distortion or cropping are converted by the broadcaster toletterbox format, to enable display on a conventional 4×3 receiverscreen. The broadcaster effectively adds blank or matte top and bottomborders, and broadcasts the combined picture and borders in a 4×3composite image signal. Assuming standard 16×9 and 4×3 ratios, only 181horizontal lines in each field are devoted to the main video, theremaining lines being the matte, gray or black borders. The borders areakin to a picture from a second video source, simultaneously displayedwith the main picture.

It can be a complex problem to adjust format ratios, particularly ifmultiple active video sources are to be displayed at once. It may benecessary, for example, to develop consistent timing signals fromasynchronous simultaneously-displayed sources, to switch betweenmultiple sources during scanning to generate multiple picture displays,and/or to provide lower resolution compressed displays from higherresolution picture data signals.

It is an aspect of the invention automatically to adjust a means fordisplaying a video source signal so as to adaptively accommodate anyformat display ratio within a predetermined range, the active portion ofthe video source signal being expanded vertically and centered so as tofill the display screen.

It is another aspect of the invention to detect a first and last line ofactive video in the video source signal repetitively and to calculate azoom level and a pan position at which the current active video willfill the display area, for adaptively responding to changes in aspectratio.

It is also an aspect of the invention to accomplish adjustment of thevideo signal in a manner which is insensitive to spurious line readingsand blank or otherwise invalid screens.

According to an inventive arrangement, an incoming signal is convertedcontinuously and adaptively from whatever active video display formatratio, into a display format that uses all the available vertical screenarea, subject to user selection or programming. Even if the displayformat ratio changes during viewing, for example where border letteringappears or thereafter disappears in a letterbox format signal, or wherethe incoming signal changes from letterbox to conventional or back, thereceiver according to the invention recalculates the timing and displayparameters to adjust over successive fields, making optimal use of theavailable display area. The invention provides high resolution, singleand multiple picture displays from single or multiple asynchronoussources having similar or different format ratios, and with selectabledisplay format ratios, all on a continuous and adaptively ongoing basis.

A wide screen television, as described herein can have a format displayratio, for example, of 16×9. The invention provides an opportunity todisplay signals which are received in the letterbox format with greaterflexibility. Signals that were originally produced in the 16×9 aspectratio but have been converted to letterbox images (e.g., a 4×3 imagewith blanked top and bottom borders around the active video) may bezoomed or expanded to fill the screen with the active video whilemaintaining the original aspect ratio thereof.

Sources in which the active video has an aspect ratio greater than thedisplay screen ratio (e.g., a 20:9 source vs. 16:9 screen) fill thescreen vertically and are horizontally cropped. Sources having an aspectratio less than that of the screen fill the screen horizontally and arevertically cropped. Those portions of the a 4×3 broadcast signal devotedto the matte borders are automatically cropped. The correction is madeadaptively, recalculating the extent of the zoom and pan as thesituation changes.

A video display control system according to an inventive arrangementautomatically controls a video display responsive to detection ofletterbox format input video signals having varying format displayratios. A detection circuit continuously detects the first and lastlines of active video in a video signal. A memory stores scan linenumbers corresponding to the first and last lines of active video. Theheight of the picture is determined from the scan line numbers of thefirst and last lines of active video. The picture height is indicativeof the format display ratio of the letterbox input. A comparator circuitcompares the picture height to a threshold corresponding to the widestexpected format display ratio of a letterbox input signal. A controlcircuit, for example a microprocessor, is operable in a first mode ofoperation for enabling the memory to continuously update the stored scanline numbers unless the picture height exceeds the threshold, andoperable in a second mode of operation for enabling the memory to updatethe stored scan line numbers when active video is detected in a videoline corresponding to a scan line number less than the stored first lineor greater than the stored last line. At least one of picture size andpicture cropping on the display means is controlled responsive to thepicture height. A circuit for initiating the first mode of operationafter the second mode of operation can operate automatically ormanually. The system can adaptively display the input video signal inthe maximum available display area, or with maximum vertical height,regardless of the specific aspect ratio used for the letterbox, andregardless of the extent to which the borders are occupied by text orthe like.

A video display control system according to another inventivearrangement comprises a video display having a first format displayratio. A picture height circuit determines an active video pictureheight from an input video signal having a second format display ratio.A detector circuit identifies letterbox formats responsive to the activevideo picture height in the video signal and determines a format displayratio of the letterbox picture. A zoom control circuit is operable in afirst mode of operation for enlarging the picture in size to fill thedisplay substantially entirely, notwithstanding consequent cropping ofthe picture, and operable in a second mode of operation for enlargingthe picture in size to substantially fill the display vertically,notwithstanding consequent unused portions of the display. A verticalpan control circuit automatically centers the picture in both modes ofoperation. The detector can identify the format display ratio of theletterbox format picture. A circuit responsive to the identified formatdisplay ratio controls image aspect ratio distortion of the enlargedpicture. A deflection system is controllable in vertical size by avariable vertical scan rate, in horizontal size by variable horizontalvideo expansion and compression, and in pan position by varying thevertical reset in phase.

The letterbox video signal can be considered to have three regions(assuming the active video is disposed vertically in the middle, insteadof at the extreme top or bottom). These regions, denoted A, B and C, aresuch that regions A and C have no active video, or matte color videoluma levels which are less than a predetermined threshold of luma orluma variability. These borders correspond to relatively dark orfeatureless bars. Region B has active video, or at least video lumalevels which are more than the predetermined luma threshold andtypically are highly variable, corresponding to the picture between thedark bars. The respective time intervals of regions A, B and C are afunction of the specific letterbox format used, which can range forexample from 5×3 to 24×9, a popular ratio of 16×9 falling within therange.

The horizontal line time duration of regions A and C is approximately 20lines each for 16×9 letterbox format. The letterbox detector countshorizontal lines while examining the luma levels for regions A and/or C.If active video, or at least a minimum video luma level, is found inregions A and/or C, the letterbox detector provides an output signalwhich stores the present horizontal scanning line count, forrecalculating the conversion ratio that will place the initial and finallines of active video at predetermined points on the display screen,normally either at the extreme top and bottom, or at the respectiveheights which will result in the full horizontal width of the screenbeing occupied. The user preferably can choose options which theapparatus will effect in the display, for example permitting the user toselect presentation of blank borders if desired, or cropping,compression, expansion or combinations of these.

Operation of the detector is improved by responding proportionately to achange in the aspect ratio of the received signal. Once a particularaspect ratio signal is established for display of the signal, andthereafter active video is detected outside of the display area (e.g.,text for display in the letterbox borders of the incoming signal), thedisplay is changed to the new ratio over a number of successive frames.Preferably, the change is accomplished by responding incrementally orproportionally to changes in ratio such that the picture expands orcontracts as appropriate over a number of successive frames. The amountof incremental change can be related to the extent of correctionrequired to reach the new aspect ratio. For reduction of noticeablejitter or the like, a minimum line count threshold can be establishedbefore a correction is undertaken.

The uppermost and lowermost active video areas preferably are detectedby calculating two gradients for each line in the video field. Fourvalues are required to calculate the two gradients: maximum and minimumvalues of the current line, and maximum and minimum values of theprevious line. The first gradient, designated the positive gradient, isformed by subtracting the minimum value of the previous line from themaximum value of the current line. The second gradient, designated thenegative gradient, is formed by subtracting the minimum value of thecurrent line from the maximum value of the previous line. Either of thegradients may have positive or negative values depending on scenecontent, but the negative values of both gradients may be ignored. Thisis because only one gradient may be negative at a time, and themagnitude of the gradient with the positive value will always be greaterthan or equal to the magnitude of the gradient with the negative value.This simplifies the circuity by eliminating the need to calculate anabsolute value of the gradients. If either gradient has a positive valuewhich exceeds a programmable threshold, video is considered to bepresent on either the current line or on the previous line. These valuescan be used by a microprocessor to make a determination of whether ornot the video source is in the letterbox format and to determine therespective line numbers. The calculation can be performed only for thetop (or the bottom) of the image, provided means are provided forcentering the image vertically. Whereas the detection of active video isa function of gradient rather than luma absolute value, the devicetherefore detects inactive borders of a predetermined matte color, evenif the borders have a substantial (but unchanging) luma level.

Similarly, the horizontally outermost active area of the image can bedetermined by calculating gradients for successive pixels or groups ofpixels at the beginning or end of successive horizontal lines. Again,this need only be done on one side if the picture is centered.

FIGS. 1(a)-1(f) illustrate different display formats of a wide screentelevision, with FIG. 1(d) illustrating certain terms used herein.

FIG. 2 is a block diagram of a wide screen television in accordance withaspects of this invention and adapted for operation at 2f_(H) horizontalscanning.

FIG. 3 is a block diagram of a gate array for embodying the televisionaccording to FIG. 2, illustrating main, auxiliary and output signalpaths.

FIG. 4 is a combination block and circuit diagram for the deflectioncircuit shown in FIG. 2.

FIGS. 5 and 6 are diagrams useful for explaining operation of anautomatic letterbox detector.

FIG. 7 is a block diagram of an automatic letterbox detector asexplained in connection with FIGS. 5 and 6.

FIG. 8 is a block diagram of an alternative circuit for implementing anautomatic letterbox detector.

FIG. 9 is a block diagram of a vertical size control circuit includingan automatic letterbox detector.

FIG. 10 is a flowchart useful for explaining the manner in whichadaptive letterbox display is implemented according to an inventivearrangement.

FIG. 11(a) is a diagram and FIG. 11(b) is a flowchart, together usefulfor explaining the manner in which adaptive letterbox display isreimplemented after interruption

The four parts of FIG. 1 illustrate a subset of picture display formatswhich can be implemented according to different inventive arrangements.The 16×9 and 4×3 basic formats are shown superimposed on one another asone example of superimposing formats having different aspect ratios.According to the invention the formats superimposed on one another neednot be limited to any specific sizes because the apparatus according tothe invention automatically senses the active video aspect ratio andoptimizes the presentation to the size of the display.

FIG. 1(a) illustrates a television, for example direct view orprojection, having a relatively wide (e.g., 16×9) format display ratioon which a picture can be displayed. When a 4×3 picture is transmittedand displayed, as in FIG. 1(b), inactive video areas displayed asvertical black, gray or matte bars appear at the lateral sides.Similarly, when a 16×9 picture is displayed on a 4×3 format display asin FIG. 1(c), the inactive video areas occur as horizontal bars on thetop and bottom. The arrangement shown in FIG. 1(c) can be encoded in avideo signal from a source, whereupon the signal is commonly referred toas being in letterbox format. In this instance, the viewed picture isvertically smaller than the available display area, as necessary todisplay the full horizontal line information without cropping orcompressing the video information. Assuming that a 16×9 format displayratio source is converted prior to transmission, so that it will fillthe vertical extent of a viewing surface of 4×3 format display, it isclear from FIG. 1(b) that either information will be cropped from theleft and/or right sides, or optionally compressed, whereby the resultingpicture will evidence distortion by relative vertical elongation(equivalent to relative horizontal compression). It is also possible topan the displayed signal, which is a further alternative. Whereas all ofthese alternatives either omit or change the nature of the signal, noneis particularly appealing.

The automatic letterbox detection system of the invention automaticallyvertically expands and centers letterbox sources with variable aspectratios to fill a television screen vertically while maintaining theoriginal aspect ratio. Sources with aspect ratios greater than 16:9 arehorizontally cropped while those with aspect ratios less than 16:9 aredisplayed with sidebars visible. The vertical zoom is controlled byvarying the vertical scan rate; the horizontal zoom is controlled byvarying the horizontal expansion/compression; and, the vertical pan isadjusted by varying the phase of the vertical reset pulse. After theinitial zoom in, the system calculates these parameters and updates themif active video is detected in regions of the picture not displayed. Theline numbers from the detection circuit are filtered to eliminatespurious readings, and blank or invalid screens are ignored.

Referring to FIG. 1(d), the first line number (the first line of activevideo detected in the current field) is obtained from a register in theletterbox detection circuit, as is the last line number. Similarly thevideo begin line (the line number between the upper letterbox region andthe active video region) and the video end line (bordering the lowerletterbox region) are obtained. The scene height can then be calculatedas the difference between the end line number and the begin line number,and the center line number calculated as the average of the two. Thescreen center is the line number at the center of the screen, and ispreferably the spot at which the center line number will be displayed.

For eliminating errors, the height of the scene is compared to a minimumheight threshold corresponding to the greatest aspect ratio that isexpected to be encountered (typically 24:9). When this threshold isexceeded, it is assumed that the current scene is blank or the currentline numbers are invalid, and no adjustments are made based on thevalues.

For eliminating jitter, the difference between the current and previousvalues of the screen parameters are compared to a change threshold. Whenthe difference is less than the threshold, the resulting change does notwarrant updating the parameters. When updated values for the first andlast line number of active video are requested, the letterbox detectorregisters are sampled. This occurs no more than once per frame formultiple frames. The results are filtered, e.g., via a two stage medianfilter, and used to determine the borders between the active video andletterbox regions.

The system is activated by the user when a letterbox video source ispresent and it is desired to fill the screen vertically with activevideo. Once activated, scene height and center of scene are calculatedfrom the first and last line numbers of the current scene. If the sceneheight is less than the minimum height threshold, the system continuesto update scene height until the threshold is exceeded. This causes thesystem to ignore blank scenes and erroneous first and last line numbers.

A comparison of FIGS. 1(e) and 1(f) shows the practical effect of theinvention. In FIG. 1(e), active the letterbox source signal has beenzoomed by the required amount to fill the screen of a wide formatdisplay. In the embodiment shown the zoom in both the horizontal and thevertical dimensions (which are proportionately equal so as to maintainthe aspect ratio of the active video signal) is such as to exactly matchthe dimensions of the wide screen display area. It is also possible thatan active video signal having an aspect ratio greater than that of thedisplay area could be received, in which event filling the vertical areawould cause the edges of the horizontal picture to be cropped.

Should active video appear in the letterbox region, such as the stormwarning shown in FIG. 1(f), the last line number of active videochanges. As a result, the system updates the center line number and theextent of zoom such that the new last video line appears at the bottomof the display area. This operation is distinct from simply switchingfrom one format ratio to another because, as shown in FIG. 1(f), thezoom and positioning are arranged adaptively and will completely fillthe display screen as necessary to accommodate changing circumstances.The effect of the additional area of active video provided by the textis to change the aspect ratio of the active video such that the ratio isless than the ratio of the display. As a result, the vertically-fullsignal is displayed with sidebars.

A 16×9 format display ratio letterbox picture, which is carried in a 4×3format display ratio signal, can be progressively scanned by linedoubling or line addition, so as to provide a larger display withsufficient vertical resolution. A wide screen television in accordancewith this invention can display such a 16×9 format display ratio signalfrom a main source, one or more auxiliary sources, or an external RGBsource.

It is possible to make various uses of inactive video areas in adisplay. For example, the inactive areas are sometimes used for textinformation such as weather or news reporting in letterbox formatbroadcasts. The inactive areas can have a superimposed picture added,for example for previewing another channel. This invention does notconcern the use made of the inactive video areas so much as the amountof area which will be rendered inactive by the television receiver inadjusting a source having inactive areas to a display.

Data sampling limitations in the auxiliary video signal processing pathcomplicate the generation of a high resolution picture which is as largein size as the display from the main video signal. Various methods canbe developed for overcoming these complications.

An overall block diagram for a wide screen television incorporating theinventive arrangements, and adapted to operate with 2f_(H) horizontalscanning, is shown in FIG. 2 and generally designated 10. The television10 generally comprises a video signals input section 20, a chassis or TVmicroprocessor 216, a wide screen processor 30 having a wide screenprocessor 309, a 1f_(H) to 2f_(H) converter 40, a deflection circuit 50,an RGB interface 60, a YUV to RGB converter 240, kine drivers 242,direct view or projection tubes 244 and a power supply 70. The groupingof various circuits into different functional blocks is made forpurposes of convenience in description, and is not intended as limitingthe physical position, packaging or specific coupling of such circuitsrelative to one another.

The video signals input section 20 is adapted for receiving a pluralityof composite video signals from different video sources. The videosignals may be selectively switched for display as main and auxiliaryvideo signals. An RF switch 204 has two antenna inputs ANT1 and ANT2.These represent inputs for both off-air antenna reception and cablereception. The RF switch 204 controls which antenna input is supplied toa first tuner 206 and to a second tuner 208. The output of first tuner206 is an input to a one-chip controller 202, which performs a number offunctions related to tuning, horizontal and vertical deflection andvideo controls. The particular one-chip shown is industry designatedtype TA7730. The baseband video signal VIDEO OUT developed in theone-chip and resulting from the signal from first tuner 206 is an inputto both video switch 200 and the TV1 input of wide screen processor 30.Other baseband video inputs to video switch 200 are designated AUX1 andAUX2. These might be used for video cameras, laser disc players, videotape players, video games and the like. The output of the video switch200, which is controlled by the chassis or TV microprocessor 216 isdesignated SWITCHED VIDEO. The SWITCHED VIDEO is another input to widescreen processor 30.

A switch SW1 of wide screen processor 30 selects between the TV1 andSWITCHED VIDEO signals to choose the source for the SELECTED COMP OUTvideo signal which is an input to a Y/C decoder 210. The Y/C decoder 210may be implemented as an adaptive line comb filter. Two further videosources S1 and S2 are also inputs to the Y/C decoder 210. S1 and S2designate different S-VHS sources, and each consists of separateluminance and chrominance signals. A switch, which may be incorporatedas part of the Y/C decoder as in some adaptive line comb filters, orwhich may be implemented as a separate switch, is responsive to the TVmicroprocessor 216 for selecting one pair of luminance and chrominancesignals as outputs designated Y₋₋ M and C₋₋ IN respectively. Theselected pair of luminance and chrominance signals is thereafterconsidered the main signal and is processed along a main signal path.Signal designations including ₋₋ M or ₋₋ MN refer to the main signalpath. The chrominance signal C₋₋ IN is redirected by the wide screenprocessor back to the one-chip 202, for developing color differencesignals U₋₋ M and V₋₋ M. In this regard, U is an equivalent designationfor (R-Y) and V is an equivalent designation for (B-Y). The Y₋₋ M, U₋₋M, and V₋₋ M signals are converted to digital form in the wide screenprocessor for further signal processing.

The second tuner 208, functionally defined as part of the wide screenprocessor 30, develops a baseband video signal TV2. A switch SW2 selectsbetween the TV2 and SWITCHED VIDEO signals for input to Y/C decoder 220.The Y/C decoder 220 may be implemented as an adaptive line comb filter.Switches SW3 and SW4 select between the luminance and chrominanceoutputs of Y/C decoder 220 and the luminance and chrominance signals ofan external video source, designated Y₋₋ EXT and C₋₋ EXT respectively.The Y₋₋ EXT and C₋₋ EXT signals correspond to the S-VHS input S1. TheY/C decoder 220 and switches SW3 and SW4 may be combined, as in someadaptive line comb filters. The output of switches SW3 and SW4 isthereafter considered the auxiliary signal and is processed along anauxiliary signal path. The selected luminance output is designated Y₋₋A. Signal designations including ₋₋ A, ₋₋ AX and ₋₋ AUX refer to theauxiliary signal path. The selected chrominance is converted to colordifference signals U₋₋ A and V₋₋ A. The Y₋₋ A, U₋₋ A and V₋₋ A signalsare converted to digital form for further signal processing. Thearrangement of video signal source switching in the main and auxiliarysignal paths maximizes flexibility in managing the source selection forthe different parts of the different picture display formats.

A composite synchronizing signal COMP SYNC, corresponding to Y₋₋ M, isprovided by the wide screen processor to a sync separator 212. Thehorizontal and vertical synchronizing components H and V respectivelyare inputs to a vertical countdown circuit 214. The vertical countdowncircuit develops a vertical reset signal which is directed into the widescreen processor 30. The wide screen processor generates an internalvertical reset output signal INT VERT RST OUT directed to the RGBinterface 60. The RGB interface 60 includes switching means whichselects between the internal vertical reset output signal and thevertical synchronizing component of the external RGB source to produce aselected vertical synchronizing component SEL₋₋ VERT₋₋ SYNC, which isdirected to the deflection circuit 50. Horizontal and verticalsynchronizing signals of the auxiliary video signal are developed bysync separator 250 in the wide screen processor.

The 1f_(H) to 2f_(H) converter 40 is responsible for convertinginterlaced video signals to progressively scanned noninterlaced signals,for example wherein each horizontal line is displayed twice, or anadditional set of horizontal lines is generated by interpolatingadjacent horizontal lines of the same field. In some instances,selective use of a previous line or an interpolated line will dependupon the level of movement which is detected between adjacent fields orframes. The converter circuit 40 operates in conjunction with a videoRAM 420. The video ram may be used to store one or more fields of aframe, to enable the progressive display. The converted video data,shown in FIG. 2 as signals Y₋₋ 2f_(H), U₋₋ 2f_(H) and V₋₋ 2f_(H), issupplied to the RGB interface 60.

The RGB interface 60 enables selection of the converted video data orexternal RGB video data for display by the video signals input section.The external RGB signal is deemed to be a wide format display ratiosignal adapted for 2f_(H) scanning. The vertical synchronizing componentof the main signal is supplied to the RGB interface by the wide screenprocessor as INT VERT RST OUT, enabling a selected vertical synccorresponding to the main video or the external video to be available tothe deflection circuit 50. Operation of the wide screen televisionenables user selection of an external RGB signal, by generating aninternal/external control signal INT/EXT. However, the selection of anexternal RGB signal input, in the absence of such a signal, can resultin vertical collapse of the raster, and potential damage to the cathoderay tube or projection tubes. Accordingly, the RGB interface circuitdetects the presence of an external synchronizing signal, and overridesselection of a non-existent external RGB input to avoid such damage. TheWSP microprocessor (WSP μP 309) also supplies color and tint controlsfor the external RGB signal.

The wide screen processor 30 can include a picture-in-picture processor301 for special signal processing of the auxiliary video signal. Theterm picture-in-picture is sometimes abbreviated as PIP or pix-in-pix.For this function a gate array 300 combines the main and auxiliary videosignal data in a wide variety of possible display formats, for examplewith the auxiliary signal displayed on an area of the main display or inthe inactive video border regions. The picture-in-picture processor 301and gate array 300 are under the control of the wide screenmicroprocessor (WSP μP). The WSP microprocessor is responsive to the TVmicroprocessor 216 over a serial bus. The serial bus includes signalpaths for data, clock signals, enable signals and reset signals. Thewide screen processor 30 also generates a composite verticalblanking/reset signal, as a three level sandcastle signal.Alternatively, the vertical blanking and reset signals can be generatedas separate signals. A composite blanking signal is supplied by thevideo signal input section to the RGB interface.

The deflection circuit 50, shown in more detail in FIG. 4, receives avertical reset signal from the wide screen processor, a selected 2f_(H)horizontal synchronizing signal from the RGB interface 60 and additionalcontrol signals from the wide screen processor. These additional controlsignals relate to horizontal phasing, vertical size adjustment andeast-west pincushion correction. The deflection circuit 50 supplies2f_(H) flyback pulses to the wide screen processor 30, the 1f_(H) to2f_(H) converter 40 and the YUV to RGB converter 240.

Operating voltages for the entire wide screen television are generatedby a power supply 70 which can be energized by an AC mains supply.

With further reference to FIG. 2, the principal components of the widescreen processor are a gate array 300, a picture in picture circuit 301,analog to digital and digital to analog converters (not shown), thesecond tuner 208, a wide screen processor microprocessor and a widescreen output encoder. The gate array 300 and the included signal pathsand functions are shown in more detail in FIG. 3. The gate array 300 isresponsible for combining video information from the main and auxiliarysignal paths to implement a composite wide screen display, for exampleone having a picture in picture or other formatting of plural sourcesfor display on the screen. Clocking information for the gate array canbe provided by a phase locked loop tracking the 1f_(H) signal of theselected source. The main video signal is supplied to the wide screenprocessor in analog form, and YUV format, as signals designated Y₋₋ M,U₋₋ M and V₋₋ M from the Y/C decoder 210 and the one-chip 202, shown inFIG. 2. These signals are converted from analog to digital form fordigital manipulation as necessary, and later are converted from digitalto analog for reading out the required video information, in knownmanner using A to D and D to A converters (not shown).

The color component signals are referred to by the generic designationsU and V, which may be assigned to either R-Y or B-Y signals, or I and Qsignals. The sampled luminance bandwidth is limited to 8 MHz because thesystem clock rate is 1024f_(H), or approximately 16 MHz. A single analogto digital converter and an analog switch can be multiplexed to samplethe color component data because the U and V signals are limited to 500KHz, or 1.5 MHz for wide I. A start of line pulse can be generated tosynchronize the multiplexing arrangement at the beginning of eachhorizontal video line, thereafter toggling during the horizontal line,with the Y and UV data streams being shifted to correctly pair thesamples. Each UV pair represents one vector and must be paired with thecorresponding V element of the same vector to prevent a color shift.

The PIP circuit and/or the gate array may include means for enhancingthe resolution of the auxiliary data to be superimposed on the picturenotwithstanding the data compression. A number of data reduction anddata restoration schemes have been developed, including for examplepaired pixel compression and dithering and dedithering. Moreover,different dithering sequences involving different numbers of bits anddifferent paired pixel compressions involving different numbers of bitsare contemplated. One of a number of particular data reduction andrestoration schemes can be selected by the WSP μP 309 in order tomaximize resolution of the displayed video for each particular kind ofpicture display format.

The gate array includes interpolators which operate in conjunction withline memories, which may be implemented as first in first out memoriesor FIFO's 356 and 358. The interpolator and FIFO's are utilized toresample the main signal as desired. An additional interpolator canresample the auxiliary signal. Clock and synchronizing circuits in thegate array control the data manipulation of both the main and auxiliarysignals, including the combination thereof into a single output videosignal having Y₋₋ MX, U₋₋ MX and V₋₋ MX components. These outputcomponents are converted to analog form by digital to analog converters(not shown) associated with 1f_(H) to 2f_(H) converter 40. The analogform signals, designated Y, U and V, are converted by 1f_(H) to 2f_(H)converter 40 to noninterlaced scanning.

For adjusting the aspect ratio of the displayed signal to take advantageof the available vertical or horizontal display span in accordance withuser requirements, operation of the deflection circuit is adjusted asneeded. The deflection circuit 50 is shown in more detail in FIG. 4. Acircuit 500 is provided for adjusting the vertical size of the raster,in accordance with a desired amount of vertical overscan necessary forimplementing different display formats. As illustrated diagrammatically,a constant current source 502 provides a constant current I_(RAMP) whichcharges a vertical ramp capacitor 504. A transistor 506 is coupled inparallel with the vertical ramp capacitor, and periodically dischargesthe capacitor responsive to the vertical reset signal. In the absence ofany adjustment, current I_(RAMP) provides the maximum available verticalsize for the raster. This might correspond to the extent of verticaloverscan needed to fill the wide screen display by an expanded 4×3format display ratio signal source. To the extent that less verticalraster size is required, an adjustable current source 508 diverts avariable amount of current I_(ADJ) from I_(RAMP), so that vertical rampcapacitor 504 charges more slowly and to a smaller peak value. Variablecurrent source 508 is responsive to a vertical size adjust signal. Thissignal is presented in analog form, generated by vertical size controlcircuit 1030 shown in FIG. 9, whereby the vertical size is automaticallyadjusted over a continuous range rather than switched abruptly from onesize to another. Vertical size adjustment 500 is independent of a manualvertical size adjustment 510, which may be implemented by apotentiometer or back panel adjustment knob. In either event, thevertical deflection coil(s) 512 receive(s) driving current of the propermagnitude. Horizontal deflection adjustment is likewise provided. Thehorizontal size is changed by phase adjusting circuit 518, East-West pincorrection circuit 514, a 2f_(H) phase locked loop 520 and horizontaloutput circuit 516.

The RGB interface circuit 60 is shown generally in FIG. 2. The signalwhich is to be ultimately displayed will be selected between the outputof the 1f_(H) to 2f_(H) converter 40 and an external RGB input. Forpurposes of the wide screen television described herein, the externalRGB input is presumed to be a wide format display ratio, progressivelyscanned source. A composite blanking signal is also available from thevideo signals input section 20. The external 2f_(H) compositesynchronizing signal for the external RGB signal is coupled through togenerate an external sync detect signal coupled to wide screen processor30. Selection of internal or external synchronization is generated bythe WSP μP 309 and signalled to the RGB interface via the INT/EXT line.Selection of internal or external video sources is a user selection.However, if a user inadvertently selects an external RGB source, when nosuch source is in fact operational, an override control signal preventsselection of the external RGB source. The RGB interface 60 also receivestint and color control signals from the WSP μP 309.

The wide screen television can be arranged to expand or compress videoas well as to adjust the format ratio by selecting an amount ofoverscan, as a user selectable option. Similarly, special effects suchas picture-in-picture effects requiring an adjustment of the videoresolution can be accomplished. The picture-in-picture processorincludes analog-to-digital conversion means, timing and control circuitsand a digital-to-analog converter section for adjustments to resolution,being accomplished together with corresponding deflection adjustments toachieve a desired effect.

A known picture-in-picture processor is the CPIP chip developed byThomson Consumer Electronics, Inc., as described a publication entitledThe CTC 140 Picture in Picture (CPIP) Technical Training Manual,available from Thomson Consumer Electronics, Inc., Indianapolis, Ind. Ina single picture mode, or in a picture-in-picture mode, it is possibleto enable a user to alter the relative sizes of a plurality of picturesbeing displayed, for example to zoom in on a selected portion of asingle picture, for example, in steps. While in the zoom mode a user maysearch or pan through the picture, display the small, large or zoomedpicture in freeze frame (still picture) format, etc.

The picture-in-picture processor can asymmetrically compress video datain one of a plurality of selectable display modes. In one mode ofoperation, for example, the pictures are compressed 4:1 in thehorizontal direction and 3:1 in the vertical direction. Asymmetriccompression of course produces aspect ratio distorted pictures as storedin the video RAM. However, if these pictures are read out normally, asfor example in the channel scan mode, for display of a 16×9 formatdisplay ratio screen, the pictures appear correct. The picture fills therequired area of the screen and there is no aspect ratio distortion.Asymmetric compression by the picture-in-picture processor makes itpossible to generate the special display formats on a 16×9 screenwithout external speed up circuitry.

For expansion and compression, including the adjustment of thedeflection circuit and the resolution in connection with a main andauxiliary (e.g., PIP) signal, the main signal path 304, auxiliary signalpath 306 and output signal path 312 of the gate array 300 are shown inblock diagram form in FIG. 3. The gate array also comprises aclocks/sync circuit 320 and a WSP μP decoder 310. Data and addressoutput lines of the WSP μP decoder 310, identified as WSP DATA, aresupplied to each of the main circuits and paths identified above, aswell as to the picture-in-picture processor and resolution processingcircuits. In the example as shown in FIG. 2 the PIP 301 and otherelements are shown as separate from the gate array 300 in the widescreen processor 30. It will be appreciated that whether or not certaincircuits are, or are not, defined as being part of the gate array islargely a matter of convenience for facilitating explanation of theinventive arrangements, because it is readily possible to eitherincorporate additional functions in a gate array or to remove certainfunctions to be accomplished by circuits apart from the gate array.

The gate array is responsible for expanding, compressing and croppingvideo data of the main video channel, as and if necessary, to implementdifferent picture display formats. Referring to FIG. 3, the luminancecomponent Y₋₋ MN is stored in a first in first out (FIFO) line memory356 for a length of time depending on the nature of the interpolation ofthe luminance component. The combined chrominance components U/V₋₋ MNare stored in FIFO 358. Auxiliary signal luminance and chrominancecomponents Y₋₋ PIP, U₋₋ PIP and V₋₋ PIP are developed by demultiplexer355. The luminance component undergoes resolution processing, as desiredand as discussed above, in circuit 357, and is expanded as necessary byinterpolator 359, generating signal Y₋₋ AUX as an output.

In some instances, the auxiliary display will be as large as the mainsignal display. Memory limitations associated with thepicture-in-picture processor and video RAM 350 can result in aninsufficient number of data points or pixels for filling a large displayarea. In those circumstances, resolution processing circuit 357 can beused to restore pixels to the auxiliary video signal, i.e. to insertpixels for those lost during data compression, or reduction. Insertedpixels can be repeated or can be defined by the values for severalneighboring pixels. Dithering/dedithering arrangements can also beeffected.

The auxiliary channel is sampled at 640f_(H) rate while the main channelis sampled at a 1024f_(H) rate. The auxiliary channel FIFO 354 convertsthe data from the auxiliary channel sample rate to the main channelclock rate. In this process, the video signal undergoes an ##EQU1##compression. This is more than the ##EQU2## compression necessary tocorrectly display the auxiliary channel signal. Therefore, the auxiliarychannel must be expanded by the interpolator 359 to correctly display a4×3 small picture. The interpolator 359 is controlled by interpolatorcontrol circuit 371, which is itself responsive to WSP μP 340. Theamount of interpolator expansion required is ##EQU3## The expansionfactor X is determined as follows: ##EQU4##

The chrominance components U₋₋ PIP and V₋₋ PIP are delayed by circuit367 for a length of time depending on the nature of the interpolation ofthe luminance component, generating signals U₋₋ AUX and V₋₋ AUX asoutputs. The respective Y, U and V components of the main and auxiliarysignals are combined in respective multiplexers 315, 317 and 319 in theoutput signal path 312, by controlling the read enable signals of theFIFO's 354, 356 and 358. The multiplexers 315, 317 and 319 areresponsive to output multiplexer control circuit 321. Output multiplexercontrol circuit 321 is responsive to a clock signal, a start of linesignal, a horizontal line counter signal, the vertical blanking resetsignal and the output of the fast switch from the picture-in-pictureprocessor and WSP μP 340. The multiplexed luminance and chrominancecomponents Y₋₋ MX, U₋₋ MX and V₋₋ MX are supplied to respectivedigital/analog converters 360, 362 and 364 respectively, followed by lowpass filters. The various functions of the picture-in-picture processor,the gate array and the data reduction circuit are controlled by the WSPμP. The WSP μP is responsive to the TV μP 216, being connected theretoby a serial bus. The serial bus may be a four wire bus as shown, havinglines for data, clock signals, enable signals and reset signals. The WSPμP communicates with the different circuits of the gate array through aWSP μP decoder 310.

In one case, it is necessary to compress the 4×3 NTSC video by a factorof 4/3 to avoid aspect ratio distortion of the displayed picture. In theother case, the video can be expanded to perform horizontal zoomingoperations usually accompanied by vertical zooming. Horizontal zoomoperations up to 33% can be accomplished by reducing compressions toless than ##EQU5## A sample interpolator is used to recalculate theincoming video to a new pixel positions because the luminance videobandwidth, up to 5.5 MHz for S-VHS format, occupies a large percentageof the Nyquist fold over frequency, which is 8 MHz for a 1024f_(H)clock.

As shown in FIG. 3, the luminance data Y₋₋ MN is routed through aninterpolator 337 in the main signal path 304 which recalculates samplevalues based on the compression or the expansion of the video. Thefunction of the switches or route selectors 323 and 331 is to reversethe topology of the main signal path 304 with respect to the relativepositions of the FIFO 356 and the interpolator 337. In particular, theseswitches select whether the interpolator 337 precedes the FIFO 356, asrequired for picture compression, or whether the FIFO 356 precedes theinterpolator 337, as required for picture expansion. The switches 323and 331 are responsive to a route control circuit 335, which is itselfresponsive to the WSP μP. The auxiliary video signal is compressed forstorage in a video RAM, and only expansion is necessary for practicalpurposes. Accordingly, no comparable switching is required in theauxiliary signal path.

In order to implement video compressions through the use of a FIFO, forexample, every fourth sample can be inhibited from being written intothe FIFO 356. This constitutes a ##EQU6## compression. It is thefunction of the interpolator 337 to recalculate the luminance samplesbeing written into the FIFO so that the data read out of the FIFO issmooth, rather than jagged. Expansions may be performed in exactly theopposite manner as compressions. In the case of compressions the writeenable signal has clock gating information attached to it in the form ofinhibit pulses. For expanding data, the clock gating information isapplied to the read enable signal. This will pause the data as it isbeing read from the FIFO 356. In this case it is the function of theinterpolator 337, which follows the FIFO 356 during this process, torecalculate the sampled data from jagged to smooth. In the expansioncase the data must pause while being read from the FIFO 356 and whilebeing clocked into the interpolator 337. This is different from thecompression case where the data is continuously clocked through theinterpolator 337. For both cases, compression and expansion, the clockgating operations can easily be performed in a synchronous manner, thatis, events can occur based on the rising edges of the system clock1024f_(H).

Interpolation of the auxiliary signal takes place in the auxiliarysignal path 306. The PIP circuit 301 manipulates a 6 bit Y, U, V, 8:1:1field memory in a video RAM to store incoming video data. The output ofthis video RAM is coupled to 4 to 8 bit converter 352 in FIG. 3. Thevideo RAM holds two fields of video data in a plurality of memorylocations. Each memory location holding eight bits of data. In each8-bit location there is one 6-bit Y (luminance) sample (sampled at640f_(H)) and 2 other bits. These two other bits can be used to holdpart of a U or V sample (sampled at the lower rate of 80f_(H)).Alternatively or additionally, the bits can be used to indicate whichtype of field was written into the video RAM. Since there are two fieldsof data stored in the video RAM, and the entire video RAM is read duringthe display period, both fields are read during the display scan. ThePIP circuit 301 thus can determine which field will be read out of thememory to be displayed. The PIP circuit always reads the opposite fieldtype that is being written to overcome a motion tear problem. If thefield type being read is the opposite type than that being displayed,then the even field stored in the video RAM is inverted by deleting thetop line of the field when the field is read out of memory. The resultis that the small picture maintains correct interlace without a motiontear.

The clocks/sync circuit 320 generates read, write and enable signalsneeded for operating FIFOs 354, 356 and 358. The FIFOs for the main andauxiliary channels are enabled for writing data into storage for thoseportions of each video line which is required for subsequent display.Data is written selectively from one of the main or auxiliary channels,but not both, as necessary to present data from each source on the samevideo line or lines of the display. The FIFO 354 of the auxiliarychannel is written synchronously with the auxiliary video signal, but isread out of memory synchronously with the main video signal. The mainvideo signal components are read into the FIFOs 356 and 358synchronously with the main video signal, and are read out of memorysynchronously with the main video. How often the read function isswitched back and forth between the main and auxiliary channels is afunction of the particular special effect chosen.

Generation of different special effects such as cropped side-by-sidepictures are accomplished through manipulating the read and write enablecontrol signals for the line memory FIFOs. For example in the case ofcropped side-by-side displayed pictures, the write enable control signalFIFO 354 of the auxiliary channel is active for a portion of the displayactive line period to effect cropping.

In each of the FIFOs, the video data is buffered to be read out at aparticular point in time. The active region of time where the data maybe read out from each FIFO is determined by the display format chosen.In a side-by-side cropped mode, the main channel video can be displayedon the left hand half of the display and the auxiliary channel video onthe right hand half. The arbitrary video portions of the waveforms aredifferent for the main and auxiliary channels. The read enable controlsignal of the main channel FIFOs is thus active for 50% of the displayactive line period of the display beginning with the start of activevideo (immediately following the video back porch). The auxiliarychannel read enable control signal is then active for the other 50% ofthe display active line period beginning with the falling edge of themain channel enable signal and ending with the beginning of the mainchannel video front porch. The write enable control signals arepreferably synchronous with their respective FIFO input data (main orauxiliary) while the read enable control signals are synchronous withthe main channel video.

A side by side display of two signals on a wide format display ratiodisplay, for example 16×9 is another example. Most NTSC signals arerepresented in a 4×3 format, which corresponds to 12×9. Two 4×3 formatdisplay ratio NTSC pictures may be presented on the same 16×9 formatdisplay ratio display. It is necessary either to crop or squeeze thepictures by 33%. If cropping or squeezing to this extent is unacceptableto the user due to the picture loss or the aspect ratio distortionentailed, a combination of lesser proportions of cropping and distortionmay be more acceptable. Depending on user preference as selected byswitch inputs and/or programming, the ratios of picture cropping andaspect ratio distortion may be set any where in a range between thelimits of 0% and 33% so as to reach the desired results in a combinationof both. As an example, two side by side pictures may be presented as16.7% squeezed and 16.7% cropped, thereby minimizing the adverse effectsof each.

The horizontal display time for a 16×9 format display ratio display isthe same as a 4×3 format display ratio display, because both have 62.5microsecond nominal line length. An NTSC video signal must be sped up bya factor of ##EQU7## to preserve a correct aspect ratio, withoutdistortion. The ##EQU8## factor is calculated as ratio of the twodisplay formats: ##EQU9## Variable interpolators are utilized inaccordance with aspects of this invention to speed up the video signals.In the past, FIFOs having different clock rates at the inputs andoutputs have been used to perform a similar function. By way ofcomparison, if two NTSC 4×3 format display ratio signals are displayedon a single 4×3 format display ratio display, each picture must bedistorted or cropped, or some combination thereof, by 50%. A speed upcomparable to that needed for a wide screen application is unnecessary.

Generally, the video display and deflection system is synchronized withthe main video signal. The main video signal must be speeded up, asexplained above, to fill the wide screen display. The auxiliary videosignal must be vertically synchronized with the first video signal andthe video display. The auxiliary video signal can be delayed by afraction of a field period in a field memory, and then expanded in aline memory. Synchronization of the auxiliary video data with main videodata is accomplished by utilizing the video RAM 350 as a field memoryand a first in first out (FIFO) line memory device 354 for expanding thesignal.

The asynchronous nature of the read and write clocks, however, doesrequire that steps be undertaken to avoid read/write pointer collisions.Read/write pointer collisions occur when old data is read out of theFIFO before new data has an opportunity to be written into the FIFO.Read/write pointer collisions also occur when new data overwrites thememory before the old data has an opportunity to be read out of theFIFO. The size of the FIFO is related to the minimum line storagecapacity thought to be reasonably necessary to avoid read/write pointercollisions.

The picture-in-picture processor operates in such a manner that theauxiliary video data is sampled with a 640f_(H) clock locked to thehorizontal synchronizing component of the incoming auxiliary videosignal. This operation enables orthogonally sampled data to be stored inthe video RAM. Data must be read out of the video RAM at the same640f_(H) rate. The data cannot be orthogonally displayed from the videoRAM without modification due to the generally asynchronous nature of themain and auxiliary video sources. In order to facilitate synchronizationof the auxiliary signal to the main signal, a line memory withindependent write and read port clocks is disposed in the auxiliarysignal path after the output of the video RAM.

Since the reading and writing of data from the auxiliary channel FIFO isasynchronous, and the read clock rate is considerably faster than thewrite clock rate, there is the possibility of read/write pointercollisions. A read/write pointer collision occurs when a read enablesignal is received before old data, that has already been readpreviously, has been replaced by newly written data. Interlace integritymust also be preserved. A sufficiently large memory must be chosen inthe first instance in order to avoid read/write pointer collision in theauxiliary channel FIFO.

It is a particular advantage of wide format display ratio televisionsthat different aspect ratio signals, including the relatively standard16×9 letterbox signals, can expanded to fill the wide format displayratio display screen, although it may be necessary to interpolate thesignal to provide additional vertical resolution. In accordance with anaspect of the invention, an automatic and continuously variable circuitis provided to automatically detect inactive video areas characteristicof a display of one aspect ratio carried in a signal processed accordingto a different aspect ratio. For example, the invention is operable tosense the occurrence of inactive video areas at the top and bottom of aletterbox having a wide (e.g., 16×9) ratio in a signal transmitted orread out for display in a narrower (e.g., 4×3) ratio, and automaticallyto implement expansion of the narrower display ratio signal (e.g. 4×3)which includes the wider format display ratio letterbox display (e.g.,16×9) so as to use all of the available display area according to userpreferences as discussed hereinabove. Similarly, horizontal expansioncan be effected in the same manner when a wide display is available anda narrower signal is to be shown there. The expansion can be implementedusing any or all of the expansion, compression or cropping alternativesdiscussed, using the circuit according to the invention described. Theautomatic letterbox detector is explained in detail in connection withFIGS. 5-9, using the relatively standard 4×3 and 16×9 aspect ratios asnon-limiting examples.

In order to increase the vertical height of a letterbox signal (forexample a 4×3 signal having a 16×9 active area and unused top and bottombands), the vertical scan rate of display video is increased so that theblack or matte regions at the top and bottom of the picture areeliminated, or at least substantially reduced. The automatic letterboxdetector is based on the assumption that the video signal willcorrespond generally to that shown in diagram form in FIG. 5. Regions Aand C have no active video, or least video luma levels which are lessthan a predetermined luma threshold and may be, for example, a black orgray signal or an unvarying colored matte. Region B has active video, orat least video luma levels which are more than the predetermined lumathreshold and generally vary substantially in luminance, saturationand/or hue. The respective time intervals of regions A, B and C are afunction of the letterbox format, which might range, for example, from16×9 to 21×9. The time duration of regions A and C is approximately 20lines each for 16×9 letterbox format. The letterbox detector examinesthe luma levels for regions A and/or C. If active video, or at least aminimum video luma level, is found in regions A and/or C, the letterboxdetector provides an output signal, for example a logical 0, indicatinga normal 4×3 format display ratio NTSC signal source. However, if videois detected in region B, but not in regions A and C, then the video ispresumed to be a letterbox signal source. In this case, the outputsignal would be a logical 1.

Operation of the detector can be improved by hysteresis, as showndiagrammatically in FIG. 6. Once a letterbox signal at a particularactive aspect ratio has been detected, a minimum number of successivefields of nonletterbox signal, or letterbox of a different aspect ratiomay be required before the display is changed to that necessary for thenew aspect ratio signals. A minimum number of fields can be requiredbefore switching the display to a wider or narrower screen mode.

A circuit 1000 for implementing this technique is shown in FIG. 7. Thecircuit 1000 comprises a line counter 1004, a field counter 1006 and adetector circuit 1002, in which the algorithm described above isperformed to analyze the video signal. Briefly, the circuit is operableto determine the scene height by sensing for the first and last lines ofactive video, and to effect the user's selected options for cropping,compression, expansion, etc. in view of detected variations. Assumingthat the object is to expand to the full available vertical height, thefirst and last line numbers are determined, and the average of the twodefines a center line number. The letterbox detector then triggerspositioning of the center line at the vertical center of the full screenavailable area, and linearly expands the picture vertically to place thefirst and last lines at the extreme top and bottom (or other userselection). Whereas the extent of expansion is determined by the activevideo area and is not simply switched, for example between 4×3 and 16×9,the apparatus accommodates any display ratio.

The letterbox detector can be arranged to update the first and last linecount only once per frame for multiple field frames. The extent of theupdating can be limited by filtering in order to expand or contract thedisplay to a new aspect ratio slowly, i.e., over several frames, and toprevent jitter due to repetitive small changes or those induced bynoise. In order to accomplish this filtering, the apparatus can requirea change to exceed a change threshold before any response will beundertaken. The system is not affected by scenes with flat (relativelyblank) fields at the lateral edges since it only looks for active videoin the A and C letterbox regions.

A flowchart illustrating an inventive arrangement for implementing afirst mode of operation, with maximum zoom according to one of thecriteria of maximal filling or maximal vertical filling, is shown inFIG. 10. While it is also possible to sense for active video in lateralsidebars, sensing is limited in the illustrated embodiment to upper andlower areas of potentially inactive video. By selecting the expansion,compression and cropping parameters using the first and last linenumbers the apparatus has several advantages. If upon initially zoominginto a picture the zoom passes the optimum level, the system promptlycorrects itself as successive fields are evaluated. If subtitles, stormwarnings or similar supplemental information appears in the borderregions, these are detected as active video lines and the picture isadjusted to display them in a second mode of operation, in which theextent of the zoom is decreased. If the supplemental informationdisappears, the first mode of operation can be reimplemented,automatically or manually. The upper and lower borders of active video,with and without supplemental information, are shown in FIG. 11(a). Thefirst and last video lines detected and displayed are also shown. Asshown in FIG. 11(b), the system initially zooms in to fill the screenwith video, according to either criteria for filling. The picture isthen zoomed out, as much as necessary, to display supplementalinformation, if detected. If the supplemental information is no longerdetected, for a predetermined period of time, for example 10 seconds,the zoom in mode of operation can be reimplemented. Reimplementation canoccur if the last line of detected video is less than the last line ofdisplayed video (as shown in FIG. 11(b)), or if the first line ofdetected video is greater than the first line of displayed video, orboth.

The sensed first and last lines are compared to minimum/maximum allowedvalues, and the original aspect ratio is preserved in the event that thesensed values for the first and last line fall out of the allowed range.This avoids any response to occurrences such as blank scenes. For thefirst frame following activation of the apparatus the initially detectedfirst and last line numbers are simply loaded, without filtering, andthereafter adjusted as necessary with filtering applied.

Two gradients are calculated for each line in the video field todetermine lines of active video. Four values are required to calculatethe two gradients: maximum and minimum values of the current line, andmaximum and minimum values of the previous line. The first gradient,designated the positive gradient, is formed by subtracting the minimumvalue of the previous line from the maximum value of the current line.The second gradient, designated the negative gradient, is formed bysubtracting the minimum value of the current line from the maximum valueof the previous line. Either of the gradients may have positive ornegative values depending on scene content, but the negative values ofboth gradients may be ignored. This is because only one gradient may benegative at a time, and the magnitude of the gradient with the positivevalue will always be greater than or equal to the magnitude of thegradient with the negative value. This simplifies the circuitry byeliminating the need to calculate an absolute value of the gradients. Ifeither gradient has a positive value which exceeds a programmablethreshold, video is considered to be present on either the current lineor on the previous line. These values can be used by a microprocessor todetermine the first and last lines of active video and thereby todetermine the particular aspect ratio of the active area of the signal.This determination defines whether or not the video source is in theletterbox format, and enables the further calculation of the extent ofchange to the deflection circuit and the resolution circuits needed toconvert the displayed signal as required. As mentioned herein, theconversion can be defined in part by user selections regarding theextent of cropping or distortion, or both, to which the signal will besubjected.

A circuit 1010 for implementing this method of letterbox assessment ordetection is shown in block diagram form in FIG. 8. The circuit 1010comprises a luma input filter, a line maximum (max) detector 1020, aline minimum (min) detector 1022, and an output section 1024. The lumainput filter comprises finite impulse response (FIR) stages 1012 and1014 as well as adders 1016 and 1018. The letterbox detection circuit1010 operates on the digital luma data Y₋₋ IN from the wide screenprocessor. An input filter is utilized in order to improve noiseperformance and make detection more reliable. The filter is essentiallytwo cascaded FIR stages, having a transfer function as follows:

    H(z)=1/4*(1+Z.sup.-1)*(1+Z.sup.-3).

The output of each stage is truncated to eight bits (divided by two) tomaintain a DC gain of one.

The line max detector 1020 includes two registers. The first registercontains the maximum pixel value (max pix) at the current point in theline period. It is initialized at the beginning of every line period bya one clock wide pulse designated SOL (Start of Line) to a value of 80h. The value of 80 h represents the minimum possible value for an eightbit number in two's complement format (the most significant bit beingthe sign). The circuit is enabled by a signal, designated LTRBX EN,which goes high for approximately 70% of the active video line. Thesecond register contains the maximum pixel value (max line) for theentire previous line, and is updated once per line period. Incoming lumadata Y₋₋ IN is compared to the current maximum pixel value stored in themax pix register. If it exceeds the register value, the max pix registeris updated on the next clock cycle. At the end of the video line, maxpix will contain the maximum value over the entire portion of the linefor which it was enabled. At the beginning of the next video line, thevalue of the max pix register is loaded into the max line register andthe register is reloaded with 80 h.

The line minimum detector 1022 works in an identical manner except thatthe min line register will contain the minimum pixel value for theprevious line. The min pix value is initialized to a value of 7 Fh,which is the maximum possible pixel value for an eight bit number in thetwo's complement format.

The output section 1024 will take the max line register value and themin line register value, and store them in eight bit latches that areupdated once per line. Two gradients are then calculated, namely thepositive gradient and the negative gradient. On the first line in afield where either of these gradients is positive and greater than theprogrammable threshold, an enable signal is generated which allows afirst line register to be loaded with the current line count value. Onevery line where either of the gradients is positive and exceeds theprogrammable threshold, another enable signal is generated which allowsa last line register to be loaded with the current line count value. Inthis manner the last line register will contain the last line in thefield where the threshold was exceeded. Both of these enable signals areonly allowed to occur between lines 24 and 250 in each field. Thisavoids false detections based on closed captioning information and onVCR head switching transients. At the beginning of every field, thecircuit is reinitialized, and the values in the first line and last lineregisters are loaded into respective letterbox end registers. TheLTRBX₋₋ BEG and LTRBX₋₋ END signals mark the beginning and endrespectively of a letterbox signal.

FIG. 9 illustrates an automatic letterbox detector as part of a verticalsize control circuit 1030. The vertical size control circuit comprises aletterbox detector 1032, a vertical display control circuit 1032 and a3-state output device 1034. In accordance with an inventive arrangement,the automatic letterbox detection circuit can automatically implementvertical zoom or expansion as required, for example to expand a 4×3format display ratio signal which includes the 16×9 format display ratioletterbox display, or to expand other format display ratio signals asdetected. When the output signal VERTICAL SIZE ADJ becomes active, thevertical deflection height is increased by the ratio required to employthe desired vertical height, typically the full vertical height. Thisenables the 16×9 active video portion of the letterbox portion of a 4×3signal, for example, to fill a 16×9 wide screen without image aspectratio distortion. Insofar as the display screen or the respective formatratios may have other specific values, the user's selections can definecropping or distortion particulars to display the scene as desired.

The automatic letterbox detector may rely wholly on the detection ofactive video lines, or additionally may comprise a circuit for decodinga code word or signal carried by a letterbox signal source whichidentifies the signal as letterbox format. This enables the system torespond precisely to standardized letterbox formats while preserving theability to correct non-standard sizes as well.

The system as described can display quite a number of variations ofmultiple source or altered (cropped, expanded, etc.) signal display. Theprogramming of WSP μP 309 can include a number of preset defaultarrangements wherein the system displays combinations of signals orsignals altered in the manners described Alternatively, or in addition,the user can selectively format the display using on-screen programmingtechniques or the like, as discussed for example with reference to theselection of individual sources for display. Any or all of the multiplesources can be provided in a particular aspect ratio (including aletterbox format) that does not involve active video in the same aspectratio as the area devoted to that source in the display or the compositemultiple source display. The WSP μP 309 can be arranged to make the samesort of expansion, compression or other alterations necessary to providethe desired display for the respective source. This requires simply thatthe lines of active video be sensed for each source and placed at thedisplay positions required to adaptively position the active video inthe display area.

What is claimed is:
 1. A video display control system, comprising:avideo display means having a first format display ratio; meansresponsive to a video signal for identifying when said video signal hasa letterbox format in which a picture is represented by an active videoportion and upper and lower regions bordering said picture arerepresented by substantially inactive video portions; means fordetermining a format display ratio of said picture when said letterboxformat is identified; means operable in a first mode of operation forenlarging said picture in size to fill said display means substantiallyentirely, notwithstanding consequent cropping of said picture, andoperable in a second mode of operation for enlarging said picture insize to substantially fill said display means vertically,notwithstanding consequent unused portions of said display means; andmeans for selecting one of said first and second modes of operation whensaid letterbox format is identified.
 2. The system of claim 1,comprising means for vertically centering said picture on said displaymeans in both of said modes of operation.
 3. The system of claim 1,wherein said means responsive to a video signal for identifying whensaid video signal has a letterbox format comprises means for identifyingthe first and last scan lines of said active video portion defining saidpicture, from which a picture height can be determined, said formatdisplay ratio of said letterbox format picture being related to saidpicture height.
 4. The system of claim 1, wherein said said picture isenlarged in said first and second modes of operation by respectivefactors, each of said factors being related to said format display ratioof said picture.
 5. A video display control system, comprising:a videodisplay means having a first format display ratio; means responsive to avideo signal for identifying when said video signal has a letterboxformat in which a picture is represented by an active video portion andupper and lower regions bordering said picture are represented bysubstantially inactive video portions; means for determining a formatdisplay ratio of said picture when said letterbox format is identified;means operable when said letterbox format is identified for enlargingsaid picture in size to fill said display means substantially entirely,notwithstanding consequent horizontal cropping at the sides of saidpicture.
 6. The system of claim 5, comprising means for verticallycentering said picture on said display means.
 7. The system of claim 5,comprising means responsive to at least one of said identifying meansand said determining means for controlling image aspect ratio distortionof said enlarged picture.
 8. The system of claim 5, wherein said pictureenlarging factor controls at least one of vertical picture size,horizontal picture size and image aspect ratio distortion.
 9. A videodisplay control system, comprising:a video display means having a firstformat display ratio; means responsive to a video signal for identifyingwhen said video signal has a letterbox format in which a picture isrepresented by an active video portion and upper and lower regionsbordering said picture are represented by substantially inactive videoportions; means for determining a format display ratio of said picturewhen said letterbox format is identified; means operable when saidletterbox format is identified for enlarging said picture in size tosubstantially fill said display means vertically, notwithstandingconsequent unused portions of said display means.
 10. The system ofclaim 9, comprising means for vertically centering said picture on saiddisplay means.
 11. The system of claim 9, comprising means responsive toat least one of said identifying means and said determining means forcontrolling image aspect ratio distortion of said enlarged picture. 12.The system of claim 9, wherein said picture enlarging factor controls atleast one of vertical picture size, horizontal picture size and imageaspect ratio distortion.
 13. A video display control system,comprising:a video display means having a first format display ratio;means responsive to a video signal having a second format display ratiofor identifying when said video signal has a letterbox format in which apicture having a third format display ratio is represented by an activevideo portion and upper and lower regions bordering said picture arerepresented by substantially inactive video portions; means fordetermining said third format display ratio when said letterbox formatis identified; means for enlarging said picture in size by a factorrelated to said first and third format display ratios when saidletterbox format is identified; and, means for controlling verticalcentering of said enlarged picture on said video display means inaccordance with the vertical asymmetry between said upper and lowerregions.
 14. The system of claim 13, wherein said means for controllingcentering of said picture comprises means for identifying the first andlast scan lines of said active video portion defining said picture. 15.The system of claim 14, wherein said means for controlling centering ofsaid picture further comprises means for determining a scan linevertically centered in said picture from said identified first and lastscan lines.
 16. The system of claim 13, wherein said first formatdisplay ratio is different from said second format display ratio. 17.The system of claim 13, wherein each of said first and third formatdisplay ratios is greater than said second format display ratio.